Can you explain the difference in results between the School Accountability Growth Report and the Educator Effectiveness Growth Report?
Updated 10.10.14
The Educator Effectiveness is more inclusive of all the assessments given at a school. Please see the following for further explanation: The School-wide Accountability Growth Value and the Educator Effectiveness Growth Value are two different values. The School-wide Accountability Growth Value includes any End-of-Grade assessments in reading and math in grades 4-8, science in grades 5 and 8, and English II, Biology, and Math I. The Educator Effectiveness Growth Value includes those same assessments, as well as the K-2 Assessments, NC Final Exams and many of the Career and Technical Education Post-Assessments. The State Board of Education approved the use of the educator effectiveness growth value for use in principal and assistant principal evaluation at the request of many school administrators who wanted their Standard 8 rating to be based on a more inclusive measure of growth across the various assessments administered in their schools.
Frequently Asked Questions on EVAAS - Updated 11.6.13
This document will be posted on the EVAAS public website to be released on Thursday November 7, 2013.
Frequently Asked Questions on EVAAS - Updated Spring 2013
FAQ's Document - a very thorough document with up to date information!
Frequently Asked Questions on EVAAS - Updated Summer 2013
I've noticed our EC teachers who participated in Roster Verification are receiving school growth instead of teacher growth. Can you explain?
Even though we need the EC teachers to participate in the Roster Verification process, especially when working with a regular ed teacher, unless they have 10 students on a roster with percentages totaling for at least 6 full time students, they will not receive an individual rating for their standard 6, but will receive the school index instead. Posted 1.15.15
Where can I find the School Level Composite Report?
Last year, users had access to a report called "School-Level Composite," which provided a school-wide growth value based on any End-of-Course or End-of-Grade assessments administered at the school. The "School-Level Composite" has been renamed "School Accountability Growth." This report can be located under School Reports.
This new terminology is to differentiate it from "Educator Effectiveness School Growth," which combines data from the EOCs, EOGs, NC Final Exams, and CTE Assessments into a growth value. Per the State Board of Education's policy decisions in the spring, this value, which is inclusive of more data, will be used to determine the Standard 8 rating for school administrators and the Standard 6 rating (for illustration) for any teachers who do not have individual growth values. The "Educator Effectiveness School Growth" report will be available when we release the Teacher and Administrator Evaluation Dashboard Reports – we'll have a release date for those soon! Posted 10.25.13
How can this data be used when the tests are changing?
With the implementation of the NCE we can compare teachers/schools/district from across different subjects and by using different tests. NCE is how far above or below the average a teacher, school or district scores as compared to the rest of the state each year. No matter what the test or subject is, we are always comparing everyone to everyone else in that year. Everyone has the same information. Posted 10.29.13
What is Student Growth and how is it Measured? Follow this link to read about Student Growth, how it is measured and the difference between Student Growth and Student Proficieincy. Posted 10.30.13
Why am I missing Value-Added Data from 2012-2013?
Even if a teacher administered an assessment for which teacher-level value-added analysis was possible, there are minimum student counts required for a teacher to have an individual value-added report. Please refer to the outlined the minimum student counts required below.
EOG Assessments in Science, EOC Assesments, NC Final Exams, and CTE State Assessments
A teacher must be connected to at least ten students and the equivalent of six "full students," defined as students with 100% instructional responsibility claimed by one teacher. This point is most easily explained with examples. Let's say that an EC teacher has claimed 20 students at 10% instructional responsibility for each one. While the teacher is connected to ten students, he is only connected to the equivalent of two "full students" (20 students X 10% each = 2 full students). The teacher will NOT have a value-added score because he is connected to fewer than six "full students."
A different EC teacher has claimed 20 students at 50% instructional responsibility for each one. This teacher is connected to at least ten students, and is connected to the equivalent of 10 "full students." He will have a value-added score. Each of the students must have at least three prior test scores (in any grade/subject or course) in order to be used in the analysis.
EOG Assessments in ELA and Math Grades 4-8
A teacher must be connected to six "full students," using the same terminology as described above. At least one of the six students must have a prior test score in the same subject for the previous school year, and at least five of the students must have prior assessment data in the subject at some point in their testing histories. Posted 10.28.13
What are standard errors, percentiles, and Normal Curve Equivalents?
The Standard error is calculated for one data point, like a value-added measure for a teacher. The Standard Error is a measure of certainty around the growth measure.
Percentiles show where a data point falls in relation to the entire set of data. Percentiles help us compare data points to each other.
Normal Curve Equivalents are used instead of percentiles when the distribution of a set of data is not normally shaped.
Tool to convert percentile rank to NCEs and vice versa.
How can teacher effectiveness be positive and a student scores below the target?
EVAAS looks at progress. It is possible for students to make excellent progress and still be below a standard, if they start at a very low level. That's why it's fair to teachers regardless of whether they teach low or high performing students.
How are school composites calculated?
School composites take all the gains over grades (Math & Reading EOGs) and or school effects for EOCs into account. A standard error is calculated for the combined composite effect. Students are also taken into account who are represented in more than one subject to calculate the standard error. Then the composite effect is divided by the standard error to produce the Index value which determines the school's Effectiveness level.
What is the difference between a projection and a prediction?
North Carolina uses 2 different models in EVAAS:
1. The Predictive Model is used for EOGs in grades 5 and 8 in Science, EOCs in high school, and CTE Post-Assessments and NC Final Exams. It uses a student's prior test scores to predict where a student will be in the statewide distribution of students.
2. The Gain Model is used for EOGs in grades 3-8, Mathematics, and English Language Arts. This model uses a student's prior test scores to project where students will be in the statewide distribution of students.
How does EVAAS determine a student's projected achievement?
Projections differ from predictions. Predictions are made for tests that students have already taken. Projections are made to tests students will take in the future. Once again, students' past testing histories in all subjects are used to determine where the child is likely to score relative to all other students in the state. However, in this case, the determination makes the assumption that students will make the average progress of students who took the test we are projecting to in the most recent testing year. In other words, if we are determining projections to Algebra I, we will look at the progress of students who have just taken the Algebra I test. We will then project that students who have not yet taken the Algebra I test will make progress similar to that of students with similar testing histories who have already taken the test. We say that the projections are where the students are likely to score, if they make progress that is average.
How can we have growth measures for assessments that have never been given before?
Even if an assessment has never been given before, a first time administration will produce a mean score where 50% of the students will do better than the mean and 50% of the students will do worse than the mean. Using percentiles and NCEs, EVAAS is always considering a student's position in statewide distribution, not the scale or raw score on the assessment. For assessments that are completely new like NC Final Exams, SAS performs an analysis of the data from the first semester to determine the prediction model. After the second semester administration, all of the data are used in the model to produce value-added scores.
Can you explain the difference in results between the School Accountability Growth Report and the Educator Effectiveness Growth Report?
Updated 10.10.14
The Educator Effectiveness is more inclusive of all the assessments given at a school. Please see the following for further explanation:The School-wide Accountability Growth Value and the Educator Effectiveness Growth Value are two different values. The School-wide Accountability Growth Value includes any End-of-Grade assessments in reading and math in grades 4-8, science in grades 5 and 8, and English II, Biology, and Math I. The Educator Effectiveness Growth Value includes those same assessments, as well as the K-2 Assessments, NC Final Exams and many of the Career and Technical Education Post-Assessments. The State Board of Education approved the use of the educator effectiveness growth value for use in principal and assistant principal evaluation at the request of many school administrators who wanted their Standard 8 rating to be based on a more inclusive measure of growth across the various assessments administered in their schools.
Frequently Asked Questions on EVAAS - Updated 11.6.13
This document will be posted on the EVAAS public website to be released on Thursday November 7, 2013.Frequently Asked Questions on EVAAS - Updated Spring 2013
FAQ's Document - a very thorough document with up to date information!Frequently Asked Questions on EVAAS - Updated Summer 2013
FAQ's DocumentI've noticed our EC teachers who participated in Roster Verification are receiving school growth instead of teacher growth. Can you explain?
Even though we need the EC teachers to participate in the Roster Verification process, especially when working with a regular ed teacher, unless they have 10 students on a roster with percentages totaling for at least 6 full time students, they will not receive an individual rating for their standard 6, but will receive the school index instead. Posted 1.15.15Where can I find the School Level Composite Report?
Last year, users had access to a report called "School-Level Composite," which provided a school-wide growth value based on any End-of-Course or End-of-Grade assessments administered at the school. The "School-Level Composite" has been renamed "School Accountability Growth." This report can be located under School Reports.This new terminology is to differentiate it from "Educator Effectiveness School Growth," which combines data from the EOCs, EOGs, NC Final Exams, and CTE Assessments into a growth value.
Per the State Board of Education's policy decisions in the spring, this value, which is inclusive of more data, will be used to determine the Standard 8 rating for school administrators and the Standard 6 rating (for illustration) for any teachers who do not have individual growth values. The "Educator Effectiveness School Growth" report will be available when we release the Teacher and Administrator Evaluation Dashboard Reports – we'll have a release date for those soon!
Posted 10.25.13
How can this data be used when the tests are changing?
With the implementation of the NCE we can compare teachers/schools/district from across different subjects and by using different tests.NCE is how far above or below the average a teacher, school or district scores as compared to the rest of the state each year.
No matter what the test or subject is, we are always comparing everyone to everyone else in that year. Everyone has the same information. Posted 10.29.13
What is Student Growth and how is it Measured?
Follow this link to read about Student Growth, how it is measured and the difference between Student Growth and Student Proficieincy.
Posted 10.30.13
Why am I missing Value-Added Data from 2012-2013?
Even if a teacher administered an assessment for which teacher-level value-added analysis was possible, there are minimum student counts required for a teacher to have an individual value-added report. Please refer to the outlined the minimum student counts required below.EOG Assessments in Science, EOC Assesments, NC Final Exams, and CTE State Assessments
A teacher must be connected to at least ten students and the equivalent of six "full students," defined as students with 100% instructional responsibility claimed by one teacher. This point is most easily explained with examples. Let's say that an EC teacher has claimed 20 students at 10% instructional responsibility for each one. While the teacher is connected to ten students, he is only connected to the equivalent of two "full students" (20 students X 10% each = 2 full students). The teacher will NOT have a value-added score because he is connected to fewer than six "full students."A different EC teacher has claimed 20 students at 50% instructional responsibility for each one. This teacher is connected to at least ten students, and is connected to the equivalent of 10 "full students." He will have a value-added score. Each of the students must have at least three prior test scores (in any grade/subject or course) in order to be used in the analysis.
EOG Assessments in ELA and Math Grades 4-8
A teacher must be connected to six "full students," using the same terminology as described above. At least one of the six students must have a prior test score in the same subject for the previous school year, and at least five of the students must have prior assessment data in the subject at some point in their testing histories. Posted 10.28.13What are standard errors, percentiles, and Normal Curve Equivalents?
The Standard error is calculated for one data point, like a value-added measure for a teacher. The Standard Error is a measure of certainty around the growth measure.
Percentiles show where a data point falls in relation to the entire set of data. Percentiles help us compare data points to each other.
Normal Curve Equivalents are used instead of percentiles when the distribution of a set of data is not normally shaped.
Tool to convert percentile rank to NCEs and vice versa.
How can teacher effectiveness be positive and a student scores below the target?
EVAAS looks at progress. It is possible for students to make excellent progress and still be below a standard, if they start at a very low level. That's why it's fair to teachers regardless of whether they teach low or high performing students.How are school composites calculated?
School composites take all the gains over grades (Math & Reading EOGs) and or school effects for EOCs into account. A standard error is calculated for the combined composite effect. Students are also taken into account who are represented in more than one subject to calculate the standard error. Then the composite effect is divided by the standard error to produce the Index value which determines the school's Effectiveness level.What is the difference between a projection and a prediction?
North Carolina uses 2 different models in EVAAS:
1. The Predictive Model is used for EOGs in grades 5 and 8 in Science, EOCs in high school, and CTE Post-Assessments and NC Final Exams. It uses a student's prior test scores to predict where a student will be in the statewide distribution of students.
2. The Gain Model is used for EOGs in grades 3-8, Mathematics, and English Language Arts. This model uses a student's prior test scores to project where students will be in the statewide distribution of students.
How does EVAAS determine a student's projected achievement?
Projections differ from predictions. Predictions are made for tests that students have already taken. Projections are made to tests students will take in the future. Once again, students' past testing histories in all subjects are used to determine where the child is likely to score relative to all other students in the state. However, in this case, the determination makes the assumption that students will make the average progress of students who took the test we are projecting to in the most recent testing year. In other words, if we are determining projections to Algebra I, we will look at the progress of students who have just taken the Algebra I test. We will then project that students who have not yet taken the Algebra I test will make progress similar to that of students with similar testing histories who have already taken the test. We say that the projections are where the students are likely to score, if they make progress that is average.How can we have growth measures for assessments that have never been given before?
Even if an assessment has never been given before, a first time administration will produce a mean score where 50% of the students will do better than the mean and 50% of the students will do worse than the mean. Using percentiles and NCEs, EVAAS is always considering a student's position in statewide distribution, not the scale or raw score on the assessment. For assessments that are completely new like NC Final Exams, SAS performs an analysis of the data from the first semester to determine the prediction model. After the second semester administration, all of the data are used in the model to produce value-added scores.